MAXIMUM-LIKELIHOOD NON-DECREASING RESPONSE 

ESTIMATES 

L. THOMAS RAMSEY 

^—^ , Abstract. Let Xij, 1 < i < m, 1 < i < li, be observations from a doubly- 

Cn ■ indexed sequence {Xij} of independent random variables (all of them discrete, 

or all of them absolutely continuous). Suppose that each Xij has the PDF 
f{x I 8i) from a one-parameter family of PDFs f{x \ 9). Mild assumptions are 
described under which there is a unique compound estimate </> = {<pi, . . . <f)m) 
\Q . of 6» = {6»i, . . . e^) such that 

(i) For integers i < fc in [1, rrt], </>i < tpk {4' is non-decreasing with respect to 
the index i). 
pH ^ (ii) Among all non-decreasing vectors of parameters A = (Ai, . . . , A^), 

^1 A ^ ^ ^(x I A) < ^(x I (^) 

,£^ • where 



C^ 



(a) X is the (doubly-indexed) vector of observations 

(b) I is the compound likelihood function: 

^(A I x) = n n /(^.,. I A,) 

►^ 1 An efficient algorithm is described to compute </>. The notation [a ... 6] denotes 

iy~v ' the integers in the real interval [a, b]. For J C [1 . . . m] let 

That is, /.*( J) is the sample mean of observations Xij with i E J. Here is the 
r^ ' theorem that justifies the algorithm: 

^^J , (i) Let Ti < T2 < ■ ■ ■ Ts he a. complete list of the distinct components of (f>. 

Let Ar = { J £ [f . . . m] : (j>i = Tr}. There are integers ar < fer such that 

Ar = [ar ... for]. Also, T,. = /l(j4r). 

(ii) For integers r G [1 . . . s], set 

Kr = min{ fi[ar ... A;] : A; £ [flr ■ ■ ■ m]} 

. ■ . Then, 

«^ ■ br = max{ k G [ar . . . m] : ^[a^ . . . k] = Kr } 



1. Introduction and Formal Context 

Let Xij, 1 < i < m, 1 < j < Ui, he observations from a doubly- 
indexed sequence {Xij} of independent random real variables. Suppose 



X 



Date: July 5, 201f . 

2010 Mathematics Subject Classification. Primary:62F99;Secondary:62-40,62P99,62F10. 
Key words and phrases, maximum likelihood, unimodal PDF families, non-decreasing response 
estimates. 

Mahalo nui loa to David Z. Liang. 

1 



L. THOMAS RAMSEY 



that each Xij has the PDF f{x \ 9i) from a one-parameter family of 

PDFs{/(x| ^) : eee}. 

In this note, we make compound maximum hkeUhood estimations 
6^ of 6^ = (6'i, . . . 6m), subject only to a non-decreasing constraint: 

i<k => Bi<ek 

Compound estimates 9 G 0™ that meet this constraint will be called 
non-decreasing. 

Think of each value of the index i specifiying consecutive levels of 
an explanatory variable, and 9i is the response to that i-th level of the 
explanatory variable. The goal of this note is to specify the maximum 
likelihood non- decreasing response estimate (existence and uniqueness) 
and provide an algorithm for its efficient computation. 

Of course, the same technology gives also the maximum likelihood 
non-increasing response function. One simply reverses the ordering of 
the levels of the explanatory variables, and applies the same theory and 
algorithm to the reversely ordered data. 

1.1. Formal Context (Assumptions). Let Xj.,, 1 < i < m, 1 < 
3 < ■"-«, be observations from a doubly-indexed sequence {Xij} of in- 
dependent random real variables. Suppose that each Xjj has the PDF 
f{x I 9i) from a one-parameter family 1-L of PDFs {f{x\9) : 6' G }. 
Set 

P = { X e R : (30 G 0)(/(x I 0) > 0) } 

A real number x is called observable if and only ii x eT>. 
Assumption 1. One of two cases are assumed: 
(i) The PDFs in "H are for discrete real random variables, 
(ii) The PDFs in T-L are for absolutely continuous real random vari- 
ables. 

Assumption 2. We assume that I? C and is a real interval 
of positive length. Consequently, includes the arithmetic means of 
finite sequences of observable real numbers. 

Assumption 3. Let |/s, 1 < s < t be observations from the in- 
dependent random variables Kj, 1 < s < t, with PDFs in the given 
family. As in |2] and on pages 337 - 341 of [Ij , the likelihood function 

is 

t 

L{9\y) = \[f{y.,\9) 

s=l 

where y = {yi . . .yt). We assume that, if each yi is observable and y is 
the arithmetic mean oi yi, . . ., yt, then L{9 \ y) is strictly increasing 
for 6* < y and strictly decreasing for 9 >y. 
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The next lemma is an immediate consequence of these assumptions. 
It will be applied frequently to subsets of the observations under dis- 
cussion. 

Lemma 1. Let ys E V, 1 < s < t, be observations from the indepen- 
dent random variables Ys, 1 < s < t, with PDFs in the given family. 
Then y G G and, for all rj E <d such that rj ^y 

(1) L{y I y) > L{r] \ y) 

Also, L{y I y) > 0. 

Proof, by Assumption 2, y G 0. Equation [T] follows directly from 
Assumption 3. Since is a real interval of positive length, there is 
some rj y^y in G. Therefore 

Liy I y) > L(r/ I y) > 

n 

1.2. Applicability of the Formal Context. The formal context ap- 
plies to many of the common one-parameter distributions, if one param- 
eterizes them by their means and takes care to include some boundary 
distributions. Here are some examples, verified in an appendix: 

(i) Bernoulli random variables with parameter p G [0, 1]. Note that 
we include the boundary cases and 1 because they could be 
sample means. This follows the usual practice for maximum 
likelihood estimators (see page 318 of [2]). 

(ii) Poisson random variables parameterized by their means, with 
6 = [0, cxd) (see page 644 of [1]). Note the inclusion of the 
boundary value 0. 

(iii) Let G = [0, cxd) and J-' the family of geometric random variables 
(here including one constant random variable), parameterized 
by their means (see page 644 of pLj). Given ^ G G, set p = 
1/{1 + 6). For 6 > 0, the PDF with parameter 6 is defined as 



follows: 

fix I 6) 



pYp, if X > and an integer 



otherwise 



For 9 = 0, f{x\0) = for all x except that /(O | 0) = 1. 
(iv) Normal distributions N{6, a) with a > fixed and G M. It is 
noted on page 317 of [2] that L{6 \ y) has a unique maxium at 
9 = y. To verify the slightly stronger assumptions of this note, 
one slightly modifies the argument given in [2]. 
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(v) Exponential random variables parameterized by their means 
(page 638 of P). Let 9 = (0, oo). Given 6 eQ, the PDF with 
parameter 6 is defined as follows: 

-exp(— a;/6'), for x > 

fix I e) j 

for X < 

The motivating example of the next section has binomial distribu- 
tions. The example is an application of a corresponding Bernoulli ran- 
dom variable case. The compound likelihood functions of the two sit- 
uations differ by a non-zero factor C that does not depend on any of 
the parameters Pi. C has the form 

where di is the number of successes (Is) in the Bernoulli observations 

j/j^i, . . ., Xi^m ■ 

1.3. The Algorithm. Within this formal context, and with each Xij 
observable, it will be proved that there is a unique non-decreasing com- 
pound estimate = (0i, . . . (pm) oi 9 = (6*1, . . . 9m) such that, among 
all non-decreasing A G O"*, 

A ^ ^ £(x I A) < ^(x I 0) 

where 

(i) X is the (doubly-indexed) vector of observations 
(ii) £ is the compound likelihood function: 

m Ui 

^(Aix)=nn/(^MiAo 

There is an efficient algorithm to compute 0. The notation [a . . . fe] 
denotes the integers in the [a, h\. For J C [1 . . . -m] let 

^J = ^{J) = —^ 

That is, /i( J) is the sample mean of observations Xij with i E J. 

Let Ti < T2 < . . . Ts he a complete list of the distinct components of 
0. Let Ar = {i E [1 . . .m] : 0j = r^ }. By a theorem, there are integers 
Qr < K such that Af. = [a^ . . . hr\. Also, r^ = [^{Ar). 

For integers r G [1 . . . s] , set 

(2) Kr = min{ fi[ar . . . k] : k E [ttr . . . m]} 
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It will be proved that 

(3) br = max{ A; G [a,. . . . m] : fi[ar ■ ■ ■ k] = Kr} 

Here is the algorithm: 
(i) Set ai equal to 1. Compute ki according to Equation O Then 
compute 61 according to Equation [3l For i G [ai . . .bi], set (pi 
equal to /i[ai . . . 61]. 
(ii) Proceed recursively. Suppose that {at}^^^ and {&f}[=i satisfy 
Equation |3] with ai = 1, and at = h-i + 1 for 1 < t < r. If 
br = m, the algorithm stops. If br < m, set a^+i = br + 1. 
Then compute Hr+i according to Equation [2l Then compute 
br+i according to Equation |3l For i G [ar+i,&r+i], set 0j equal 
to /iK+i. . .6r+l]- 



L. THOMAS RAMSEY 



2. A Motivating Example 



For a particular mathematics course, can one use various SAT scores 
to predict performance on the final examination? A significant subset of 
the students failed to take the final examination, in some cases because 
they withdrew from the course (with a record of W) or they simply 
received a letter grade of F. In the Data Appendix, Table |2] tabulates 
the no-show counts by SAT-R levels. The data are noisy. 

Table [1] gives the maximum likelihood non-decreasing response esti- 
mate. In Figure [Hare plotted both the observed no-show rates for each 
SAT-R score, and the maximum-likelihood non- decreasing response es- 
timate. 

On the other hand, the maximum likelihood non-increasing re- 
sponse estimate is the constant 26/152 for every SAT-R level. 

To estimate the significance of the sharp difference between the 
maximum-likelihood non-decreasing and non- increasing response es- 
timates, 10,000 data tables with the same structure as Table [2] were 
generated under a null hypothesis of a constant no-show rate of 26/152 
regardless of SAT-R score. 

• For each simulated table, the maximum-likelihood non- decreasing 
response estimate / and the maximum-likelihood non- increasing 
response estimate g were computed. This test statistic was tab- 
ulated: 

A = [ /(800) - /(330) ] - [(7(330) - g{800)] 

• The simulated quantile ranking is 0.9899 for the same statistic 
for the actual table (namely 1). That is, 101 of the simulated 
tables had A = 1 (the maximum logically possible). 

There are easier and more routine ways to reject the null hypothesis, 
but the non-decreasing response estimate itself has proved useful for 
directing resources to students who have a greater risk of failure. 

The observed table is broadly consistent with an alternative hypoth- 
esis of the maximum-likelihood non- decreasing response estimate; its 
simulated loglikelihood rank (based on 10,000 simulated tables gener- 
ated under this alternative hypothesis) is 0.3476. 
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Figure 1 . ML Non-Decreasing Response Estimate 
SAT-R Scores Total Count No-Show Count No-Show Rate 
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Overall: 152 26 17.1% 

Table 1. ML Non-Decreasing Response Estimate of No-Show Rates 



3. Existence of Likelihood-Maximizing Non-Decreasing 

Response Estimates 

Every 9 G 0"^ defines a partition S{9) of [1 . . . m] as follows. Let 
Ti, r2, . . ., Ts be a listing of the distinct components of 9 without the 
repetition of any component. Let 

S{9) = {Ai,...,As} where At = {i e[l . . .m] : 9, = n] 

Let /(^), the index of 6', be equal to the number of t G [1 . . . s] such 
that 

rt ^ li{At) 

Lemma 2. Let x = {{a^j,j}"li}i^i have each Xij G V. Suppose that 
9 G G*" is non- decreasing and has I (9) > 0. Then there is some non- 
decreasing 9 G O™ such that 

(i) S{9) has fewer members than S{9) or, S{9) = S{9) and I{9) < 
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(ii) For the compound likelihood function £{■ \ x), 

i{e |x) <£(^|x) 

Proof. Suppose first that £{9 | x) = 0. Let x = /i[l . . .m], the arith- 
metic mean of all the observations. By Lemma [H because every Xij is 
observable, we have x G 0. Let 6 G 0"^ be the constant vector with 
components equal to x. Clearly 9 is non- decreasing, S{9) = {[1 . . . m]}, 
and for all i we have ^ 

9i = x = /i[l . . .m] 
Thus I{9) = 0. By Lemma [H 

£(9\^) =L(x\^) >0 = i(9\^) 



For the rest of the proof we assume that £{9 | x) > 0. 

Let Ti < T2 . . . < Ts he a complete list of the distinct components of 
9. Set 

At = {ie[l...m] : 9i = n} 
Then 8(9) = {Ai,..., A,}. 

Because I{9) > 0, there is at least one T E [1 . . . s] such that tt ^ 

il{AT). 

Because 9 is non-decreasing, there are integers a < 6 in [1 ... m] such 
that Ax = [a ... 6] . 

Let y be the vector with ria + . . . + Uf, components {{xjj}"!]^}^^^. 
Then, 

£{9 I x) = LiTT I y) ■ C 
where 

b rii 

L{w I y) = n n •^(^*j I ^) 

i=a j=l 

and 

rii 

^= n nn/(^ 1^) 

tG[l...s\{T} JSAt j=l 

Since £{9 | x) > and C > 0, we have C > 0. 
For t G [1 . . . s]\{T} and for all i G Aj, we set 

9, = n = 9, 

For t = T, we will select some ty and for all i G At we will set 
9i = vt- Note that we will then have 

£(9\^)=L{rT\y)-C 

We break the selection of t^ into 6 cases: 
(i) ^T = i^iAi) if s = 1. 
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lllj 


Tt 


lii) 


Tt 


;iv) 


Tt 


(v) 
vi) 


Tt 
Tt 



fi{Ai) if s > 1, T = 1 and /i(Ai) < t2. 
fi{AT) a s > 1, T = s, and fi{As) > r^^i. 
tt+i ifs>l, 1<T<s and fi{AT) > tt+i- 
tt-1 ii s > 1, 1 < T < s, and //(^t) < tt-i- 
fi{AT) if s > 1, 1 < T < s, and 

Note that in every case, -pp G 9 and thus 9 G B™': 

• In cases 1, 2, 3 and 6, tt is the arithmetic mean of observable 
real numbers. By Lemma [H tt is in B. 

• In cases 4 and 5, tt is equal to a component of 9. Since 9 G G™', 
we have tt in Q. 

Next, please note that in every case tt has been carefully selected 
to make 9 be non- decreasing. 

It will be argued shortly that the change from 9i = tt to 9i = tt for 
i G At strictly increases L{- \ y). Since C > 0, that will give us 

i{9\^) = L{TT\y)-C> L{tt I y) • C = £(0 I x) 

• In cases 1, 2, 3 and 6, the change is from tt ^ I^-^At^ to /^(At). 
By Lemma [1], 

L{^{At) I y) > L[tt I y) 

• In case 4, we have tt < tt+i < fi{AT). By Assumption 3, 
L{w I y) is strictly increasing for w < fi{AT)- Therefore 

H^T+i I y) > L{tt I y) 

In case 4, tt = tt+i and thus 

L{tt I y) > L{tt I y) 

• In case 5, we have /^(At) < tt-i < tt- By Assumption 3, 
L{w I y) is strictly decreasing for w > /^(At). Therefore, 

Htt-i I y) > L{tt I y) 

In case 5, fr = tt-i and thus 

L{tt I y) > L{tt I y) 

Lastly, consider Item (i) of the Lemma. 

• In case 1, I {9) = 0. Also S{9) has one member. If S{9) has 
more than one member. Item (i) is satisfied. If S{9) has one 
member, then S{9) = S{9) = {[l...m]}. Since I{9) > by 
hypothesis, we have I (9) < I{9). That, too, satisfies Item (i). 
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• In cases 2, 3 and 6, S{9) = S{9). Also, for r y^ T, there is no 
change in whether r^ is equal to fi{Ar). However, for i G At, 
by changing for Oi = tt j^ /^(^t) to 9i = ^i^At), we have made 
1(6) = 1(6) -J. 

• In case 4, S{6) has one fewer partition member than S(6), be- 
cause 

S{9) = iSi9)\{AT,AT+i})U{AT} 
where 

At = AtU At+1 

• In case 5, again S{6) has one fewer partition member than S{6), 
because 

S{9) = {S{9)\{At^i,At}) U {A^i} 

where 

At-i = At-1 U At 

D 

Lemma 3. Let x = {{xjj}"4i}E^i have each Xij G V. Suppose that 
9 G 0™ zs non- decreasing with I{9) > 0. Then there is some non- 
decreasing 9 G 6™ with I{9) = such that 

x) < £{9 I x) 

Proof. Apply Lemma |2] to 9^^^ = 9 to get 9^^\ and continue recursively 
to apply Lemma E] to ^('^^ to get 9^''+^\ but stop if I (9^^^) = 0. 

In Lemma [21 9^'^^ has the role of 9 (provided that 1(9'^''^) > 0) and 

Q{k + 1) J^g^g ^]^g j.qIq Qf ^_ 

With each recursion 



and either 5'(^^'^+^-') has fewer members than 8(9^''^), or 
^(^(fc+i)) _ ^^^(fc)) g^j^^ 1(0(^+1)) < /(^W) 

Because S{9^^^) is finite, we can reduce the size of 3(9'^''^) at most 
finitely many times. Let K be the last time that S{9^^^) is smaller 
than S{9^^~^''). Then, after M = I{9^^'>) more steps of the recursion. 

Theorem 1. There is a non- decreasing response estimate 9 G O™' such 
that I{9) = 0, £{9 | x) > and, for all non- decreasing A G O™", 

£{X I x) < £{9 I x) 
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Proof. By Lemma [3l for any non- decreasing A G G™", there is some 
non-decreasing 6 G 0"^ with I {6) = such that 

£{X I x) < i{0 I x) 

Note that, when I{9) = 0, the partition S{9) induced by 9 determines 
9: 

ieAe S{9) -^ 9i = fi{A) 

Because [1 . . . m] is finite, there are finitely many distinct partitions 
of it. With the condition I (9) =0 imposed, that gives at most finitely 
many members of 0*" that can play the role of 9 in Lemma [31 Among 
these finitely many possibilities for 9 in Lemma [3l choose some 9 with 
i{9 I x) being largest. 

In particular, consider the A G O™' that is constant with components 
equal to x = fi[l . . .m]. Because all Xij are observable, by Lemma [T] 
we have 

i{X I x) = £(x I x) > 

It follows that 

i{9\x) >^(A |x) >0 

D 

4. Algorithm and Uniqueness 

Lemma 4. Suppose that all Xij G V. Let 9 satisfy the conclusions of 
Theorem [II Let ti < T2 < . . . < Tg be a complete listing of the distinct 
components of 9. For r E [1 . . . s], set 

Ar = {i e [I. . .m] : 9i = Tj.} 

Then 

(i) There are integers ar < K in [1 . . .m] such that Aj. = [a,, . . .br]. 
(ii) For all t G [(a^ + 1) . . .br], 

Proof. Because 9 is non- decreasing. Item (i) is immediate. 

We will prove Item(ii) by contradiction. Suppose there is r and 
t G [{ar + I) . . .br] such that 

/i[t . . . br] > fi{Ar) 

Because I{9) = 0, we have //(A^) = r,.. 

For some real rj to be chosen soon, we define 9 as follows: for i G 

[1 . . . m] let 

9i ii i < t oi i > br 
r] ii i E [t . . .br 
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If r = s, let T) = fj,[t . . .br]- If r < s, select 

Please note that this selection makes 6 be non- decreasing. Also, 77 G 0: 
(i) Because every Xij is an observable real number, r] = fi[t . . . bj] 

puts ?7 G by Lemma 1. 
(ii) Because is a real interval and both r,. and r^,+i are in 0, 

1] G {Tr,Tr+i) putS 1] E Q. 

Therefore ^ is a non-decreasing member of 0™. 
Let 

(t— 1 rii \ / m Ui 

(with empty products set equal to 1). Since C > and a factor of 
i{e I x) > 0, we have C > 0. 

By Assumption 3, with Tj. < t] < ^[t . . . 6^], 

br rii br rii 

Therefore, 

br '^i br n.i 

£(^i x) = c ■ nn^(^^^- 1 ^) > ^ • nn^(^^^- 1 ^*) = ^(^ i ^) 

«=i j = l i=t j=l 

This contradicts the likelihood maximizing property of 9 in Theorem 

m □ 

Theorem 2. Suppose that all Xij G P. Let ^ satisfy the conclusions of 
TheoremUl Let Ti < T2 < . . . < Tg be a complete listing of the distinct 
components of 9. For r E [1 . . . s], set 

Ar = {i e [I. . .m] : 9i = Tj.} 

Then 

(i) There are integers ar < br in [1 . . .m] such that Aj. = [a.^ . . .br]. 
(ii) Let 

Kr = min{ fi[ar . . .k] : fc G [a^ . . . m] } 

and set 

tr = max{ k E [ar . . .m] : fi[ar . . .k] = Kr} 
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Then tr = br- 

Proof. Item (i) is the same as Item(i) in Lemma HI and is repeated here 
to estabhsh the notation. 

Because I{6) = 0, we have r^ = fJ^iAr) for all r. 

Suppose first there is some r such that tr = hq for some q > r. Then 
fi[ar ... tr] is a convex combination with positive coefficients of r^ for 
h E [r . . .q]. For h > r, we have Th > Tj.. By the definition of k,., we 
have 

Tr = jJ^iAr) = fi[ar . . . 6,] > Kr 

Therefore, Kr > Kr. This contradiction proves that tr 7^ bq for all 
g G [r + 1 . . . s] . 

Next suppose that tr E [ag . . . {bq — 1)] for some g G [r . . . s]. By the 
definition of Hr and of tr, 

• IJ,[ar . . .bq] > Kr 

• When q > r, 

/i G [r . . . (g — 1)] =^ fi[ar ■ ■ -bh] > Hr 

In particular, ^[ar . . . bq-i] > Kr- 

We now show that fi[aq . . .tr] < Hr- li q = r this is immediate from 
the definitions of Kr and tr- Suppose that q > r. There are positive 
integers e and / such that 

, , _ eiJ.[ar ■ ■ ■ bq-l] + f^i[aq . . .tr] 

Lli\iAjy , , , l"if\ 



Therefore 

^[aq. . .tr 



e + f 

_ (e + f)fi[ar ...tr]- efi[ar . . . Vi] 

/ 

_ {e + f)Kr -efi[ar...bq_i] 

f 

= Kr + (e//) ■ {Kr - /i[a, . . . 6q_i]} 

< Kr 



because //[a,, . . . 6g_i] > Kr. 

Next we show that /i[(tr + 1) . . .bg] > Kr. There are positive integers 
e and / such that 

efi[ar ...tr] + ffijitr + 1) . . . bg] 
H[ar . . . Og\ — 
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Consequently 












fi[{tr + 1) . , 


..b,] 


(e + f)fi[ar . 


..b,] 


— efi[ar . 


. . u] 




f 










(e + f)iJ,[ar . 


..b,] 


(z^r\jy 





T, < ^^^yy^r^^,...u,,^j^.yy.r^^,...u,, ^ ^^^^^ + I) . . .h, 



Qi 



f 
= fi[ar .■■bq] + (e//) • {fl[ar ...bq]- Kr} 

> fl[aj. . . . bq] > Kr 

because fi[ar . . . bg] > Hr- 

Third, we argue that fi[(tr + 1) . . .bg] > Tg. We've shown aheady 
that 

^[ttq . . .tr] < Kr < /i[(tr + 1) • • • &</] 

There are positive integers e and / such that 

_ r , 1 _ efx[ag...tr]+ff^[{tr + l)...bq] 

Tq — H'[U'q • • • WgJ — f + f 

It follows that 

efi[{tr + 1) . . . bq] + //i [(t, + l)...bg] 

e + f 

However, this contradicts Lemma H] (since 6 satisfies the conclusions of 
Theorem [1]) . 

The only possibility left for tr is to be equal to br as desired. D 

Corollary 1. Suppose that all Xij G V. There is a unique 9 G 0™ 
that is non- decreasing and, for all non- decreasing A G 0™, 

(4) i{\ I x) < i{e I x) 

where I is the compound likelihood junction. 

Proof. By Theorem [T], there is some 6 such that satisfies the conclu- 
sions of that theorem. In particular, 6 satisfies Equation H] for all non- 
decreasing A G 9"*. 

Suppose that 9 also satisfies Equation H] for all non- decreasing A G 
0™. By Lemma [31 if 7(6') > 0, there would be some non-decreasing 
^ G 0™ such that 

"'" 'x) >i(e\^) 



So we must have I (6) =0. 

Because each Xij is observable, their arithmetic mean /x[l . . .m] is 
in 0. Let A G 0™ which has the constant component /i([l . . .m]). By 
Assumption 3, 

i{X I x) > 
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Therefore, 1(6 \ x) > 0. 

So 6 satisfies tlie conclusions of Theorem [TJ 

Note that Theorem [2] specifies S{9) = S{9) uniquely. First, the 
theorem determines Ai = [1 . . . 6i] as 6i must equal ti. Once b^. is 
determined, if 6^ < m the theorem then determines 6^+1 = tr+i- Of 
course a^+i = br + I, and thus A^+i is specified. 

However, since I{9) = I{6) = 0, we have for i G Aj. 

ei = ei = ^i{Ar) 

Thus e = e. u 

5. Examples of the Formal Context 

Throughout this section, let T be a positive integer, y : [1 . . . T] — t- D, 
and we think of each yi as an observation of a random variable Y^, with 
{Fj}^^ independent with PDFs from the given family. 

Example 1. Let G = [0, 1] and T he the family of Bernoulli random 
variables (here including two constant random variables). Given 9 ^ Q, 
the PDF with parameter 9 is defined as follows: 

if X = I 
f(x\9)= {1-9 zfx = 

z/xgR\{0, 1} 

Here "D = {0, 1}. We show that Assumption 3 holds. 
Let y have r ones and s zeros. Thus r + s = T and y = r/{r + s). 
For any A G [0, 1], 

MA):=L(A|y) = A'(l-A)^ 

Case 1: Suppose r = 0. So s = T > 0. Then 

h'ix) = sii-xr-\-i) 

For A G [0, 1), this derivative is negative. Because h{X) is continuous in 
A, the function h is strictly decreasing on [0, 1]. Since y has all zeros, 
y = 0. So Assumption 3 holds: h is strictly decreasing on [0, 1] = [y, 1] 
and strictly increasing (trivially) on [0,0]. 

Case 2: Suppose s = 0. Then r = T > and 

h\\) = rW-^ 

For A G (0, 1], this derivative is positive. Because /i(A) is continuous in 
A, the function h is strictly increasing on [0, 1]. Since y has all ones, 
y = 1. Consequently, Assumption 3 holds: h is strictly increasing on 
[0,y] = [0, 1] and h is strictly decreasing (trivially) on [1, 1]. 



' A 



16 L. THOMAS RAMSEY 

Case 3: Suppose r > and s > 0. Then 

h'{\) = ry-\l - \y + A" ■ s(l - A)^-^ 
= W''\l-\y-^[r{l-\)-s\] 

For A G (0, 1), this derivative is positive for A < rjir^s) and negative 
for A > r jir + s). Because h[X) is continuous in A, it follows that h 
is strictly increasing on [0, r j(r + s)] and strictly decreasing on \r j(r + 
s), 1]. Since y = r/{r + s), Assumption 3 holds. 

Example 2. Let = [0, oo) and T he the family of Poisson random 
variables (here including one constant random variable). Given 9 E Q, 
the PDF with parameter 9 is defined as follows: 

-J-90X 

if X >0 and an integer 



f{x\9) ' ^- 

otherwise 

Here V is the set of non-negative integers. We show that Assumption 
3 holds. 

For any A G [0, oo), let 

MA) := L(A I y) = n (e^'^) = i^e'^^A^ 

where i^ > is a factor that does not depend on A. 

Case 1. Suppose that y = 0. Then h(\) = Ke~^^ and hence 

h'{X) = -KTe-^^ 

Note that h' is negative for all A G O. Thus h is strictly decreasing on 
[0, oo) = [y, oo) and strictly increasing (trivially) on [0, 0]. 
Case 2. Suppose that y > 0. Then 

h\\) = {-KT)e-^>^\^y + Ke-^"^ ■ Ty ■ A^"^ 

= KTe-^^\^-^[-\ + y] 

Note that h' is positive on (0,y) and h' is negative on (y, oo). Since 
y > 0, we have h continuous on [0, oo) = 6. Therefore, h is strictly 
increasing on [0,y] and h is strictly decreasing on [y, oo). 

Example 3. Let = [0, oo) and T be the family of geometric random 
variables (here including one constant random variable), parameterized 
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by their means. Given 6 E Q, set p = 1/(1 + 6). For 6 > 0, the PDF 
with parameter 9 is defined as follows: 

[1 — pYp, if X > and an integer 
fix I 9) I ^ 

otherwise 

For 9 = 0, and thus p = 1, let f{x | 0) = for all x except that 
/(O I 0) = 1. 

Here V is the set of non-negative integers. 

For any A G [0, oo), set p = 1/(1 + A). Then for A > 



h{X) := L(A I y) = J] ((1 - p)» = p^{l - pf 



For A = 0, /i(0) = 1 if y is a vector of zeros and otherwise. 
Please note that h is continuous on [0, oo): 

• Suppose y = 0. Then y is a vector of zeros and /i(0) = 1. For 
A > 0, we have 

h{\)=p^{i-pY-^ = p^ = {i + \)-^ 

Clearly h is continuous on (0, oo); it is also continuous at 
because hmA^o^(A) = 1 = h{fS). 

• Suppose y > 0. Then y has at least one non-zero component 
and thus /i(0) = 0. For A > 0, 



h{\) = A^(1 + A)- 



Ty-T 



Clearly h is continuous on (0, oo). Since Ty > 0, we have 
liniAio h.{X) = = h{0). Thus h is continuous at as well. 

Case 1. Suppose that y = 0. Then y is a vector of zeros and 
h{0) = 1. For A > 0, 

h{X) = p^ = {1 + X)-^ and thus h'{X) = {-T){1 + X)'^-^ < 

Since h is continuous on [0, oo) and has a negative derivative on (0, oo), 
we know that h is strictly decreasing on [0,oo). Thus h is strictly 
decreasing on [0, oo) = [y, oo) and strictly increasing (trivially) on [0, 0]. 
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Case 2. Suppose that y > 0. For A > 0, 

, dh dp 



■i-m + xr' 

-(1 + \y'Tp^'\l - p)^'' [(1 -p)- yp] 
{I + X)-'Tp^-\l - p)^-^ [yp - {I - p)] 

[l + \)-'Tp^'\l-p)^~' 1+^ 
[1 + \r'Tp^~\l - p)"^-' 



y-x 



ii + \)ii + y)_ 

For A > 0, all the factors in the previous line are positive except for the 
factor y — X. Thus for A G (0, y), we have h'{\) > and for A G {y, oo) 
we have h'{X) < 0. Since h is continuous on [0, oo), we have h strictly 
increasing on [0,y] and strictly decreasing on \y, oo). 

Example 4. Let = R and, for a fixed a > 0, let T he the family of 
normal random variables with standard deviation a. Given 6 E Q, the 
PDF with parameter 9 is defined as follows: for all real x 

1 (-{x-ey 



fix I 6) 



exp 



2(t2 



aV27T 

Here V = 'R. We'll argue that Assumption 3 holds. 
For all real A, let 

M.,..(M.,.fl(^e.p{z(-f)!}) 



i^exp 



.(7V27r 



1=1 



2^2 



where i^ is a positive factor that does not depend on A. Then 



h'{X) = Kexp 




^T 



EUy^-xy 



2a2 
2^2 

-EL(i/.-a)^ 

2^2 



^T 



E •=! 2(y. - A) 



2^2 



T 



-TX + Y^y, 

i=l 

-X + y) 
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All factors immediately above for h'{X), except the last one, are positive 
for all A. So h' is positive for A in {—oo,y) and h' is negative for A in 
{y, oo). Since h is continuous on R, we have h is strictly increasing on 
(— oo,y] and strictly decreasing on [y, oo). 

Example 5. Let = (0, oo) and T he the family of exponential random 
variables parameterized by their means. Given 6 E Q, set r = 1/6. In 
terms of r, the PDF with parameter 9 is defined as follows: 

{re~™, for x > 
/or X < 

Here T> = (0, oo). We'll argue that Assumption 3 holds. 
Note that, since V has only positive numbers, y > 0. 
For any real ^ > and with t = 1/6, let 

h{e) := L{e I y) = n (^e^'^'O = r^e-^^ 

Then 

_ dh dr 

= {Tr^-'e--^ + r^ ■ {-Ty) ■ e"^^} ■ (-r^) 



e^ \ e 

It follows that h'{9) > if if 61 < y and h'{9) < li 9 > y. Because 
h{9) is continuous on (0, oo), we have h strictly increasing on (0,y] and 
strictly decreasing on [^, oo). 



20 





L. THOMAS RAMSEY 




R Scores 


Total Count No- Show 


330 


1 





390 


2 





400 


1 





410 


2 





420 


5 





430 


4 





440 


4 


1 


450 


3 


2 


460 


2 





470 


8 


1 


480 


11 


3 


490 


9 





500 


4 


1 


510 


11 





520 


9 





530 


8 


1 


540 


11 


4 


550 


6 


1 


560 


5 





570 


6 





580 


7 





590 


5 


1 


600 


3 


1 


610 


5 


3 


620 


4 


2 


630 







640 




2 


650 







660 







680 




1 


690 




1 


700 







710 







750 







800 




1 



Table 2. No-Show Counts for SAT-R Levels 

6. Data Appendix 
Table [2] has the no-show counts, tabulated by SAT-R scores. 
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